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ABSTRACT: Large-scale assessment programs rely on teachers to administer the tests and to 
interpret and communicate the results. Consequently, it is important that teachers have up-to-date 
knowledge about the testing programs. Education Departments’ web sites are ideal tools for 
communicating such information to teachers. The web sites of Education Departments across 
North America were surveyed for information about testing programs, including their purpose(s), 
sample items, appropriate interpretations and uses of the results, summaries of performance 
trends, and tips for communicating results to parents. Sites varied widely in what information 
was provided and the ease with which it could be located. 


Teachers are important for the successful implementation of large-scale assessments. As the first 
line of contact for students and parents, they answer questions about tests and explain the results; 
as test administrators, they control how the tests are delivered to the students. Teachers’ skills in 
administering the tests and interpreting the results are of critical importance if large-scale 
assessment programs are to be successful. To effectively develop and use these skills, teachers 
require knowledge about the testing programs. Any organization introducing or maintaining a 
testing program - usually an Education Department or a Ministry of Education (referred to 
subsequently as an ED) - needs to consider carefully what information it will provide for 
teachers and by what medium. 




Sources of Information About Student Testing 

Traditionally, EDs provided information to teachers through memoranda, information booklets, 
and training sessions. Sometimes these materials and training sessions were delivered directly to 
teachers; sometimes they were provided to principals or school or district testing coordinators, 
who then passed the information on to teachers. These approaches presented two difficulties: (a) 
the information sometimes did not reach the intended target or arrived in an altered form, and (b) 
even if the information reached the target, it might not be easily accessible when it was later 
needed. The first of these problems is sometimes addressed by the use of videos to transmit 
information efficiently and consistently across a state or province. In addition, since the mid- 
1990s, the Internet has increasingly been used to make up-to-date information about testing 
programs available on demand, helping to address the second problem. 

The Internet is now being used by most EDs to communicate broadly about testing programs. 

For example, in many states and provinces, anyone with Internet access can call up the 
standardized test results for a particular school and compare those results to results of other 
schools or groups of schools. Many states and provinces provide materials aimed at parents, 
fewer provide information specifically for teachers. 

Many individual schools and districts also have web sites. Some of these sites provide 
information about assessments. While information on these sites may be convenient, it may not 
always be up-to-date. ED sites, not local sites, are likely to be the most timely and reliable 
sources of information about large-scale assessment programs and as such were chosen as the 
focus of this study. 

This Study 

Providing information about large-scale assessment programs to teachers is critically important 
and EDs are increasingly relying on the Internet to deliver this information. But, are the ED web 
sites providing the information teachers need? This study seeks to answer two questions: (1) 
What student testing information should EDs provide? and (2) What information do ED web 
sites provide? The first question is investigated by reviewing what the research literature says 
teachers want to know about large-scale assessment programs. A survey of North American 
Education Departments’ web sites addresses the second question. 

What Student Testing Information Should EDs Provide? 

The specific teacher skills related to large-scale assessments are outlined in The Standards for 
Teacher Competence in Educational Assessment of Students (American Federation of Teachers, 
National Council on Measurement in Education, & National Education Association, 1990). To 
exercise these skills, teachers need information about the testing programs. The Principles for 
Fair Student Assessment Practices for Education in Canada (Joint Advisory Committee, 1993), 
and the Standards for Educational and Psychological Testing (American Educational Research 
Association, American Psychological Association, and National Council on Measurement in 
Education, 1999) include standards that describe the responsibilities of the organizations that 
mandate and, or develop the tests. An analysis of the three sets of standards (see Table 1) 


suggests eight categories of information teachers require about large-scale testing programs: (1) 
purpose, (2) content and format, (3) administration, (4) practice items and test-taking strategies, 
(5) consequences, (6) using and explaining results, (7) technical quality, and (8) results from 
previous years. Each of the categories is described below. Specific questions teachers might ask 
within each category are listed in Table 2. 

Purpose 

Traditionally, information from large-scale assessments was used: (a) as a supplement to or 
confirmation of information teachers already possessed, (b) as a reflection on or guide to 
instruction, and (c) to group or track students (Salmon-Cox, 1981). With the introduction of 
performance-based assessments and data-driven approaches to reform, the purposes have 
expanded: Increasingly, large-scale assessments are expected to leverage change. To make things 
more complicated, different tests from the same ED may have different purposes. For example, 
many of the current large-scale assessments are designed to facilitate comparisons of students 
and schools and tracking of performance over time. Portfolio assessments, in contrast, are best 
used to portray individual accomplishments (Cizek, 1998). 

Grant (1999) found that teachers wanted to understand the rationales for adopting specific 
assessment-based reforms. If the rationale was not clearly communicated, teachers were more 
likely to dismiss the reform as a transitory policy. In addition, understanding the purposes of the 
tests allows teachers to make independent judgments about the importance of the test purposes. 
As some teachers are pressured to increase test scores, it is particularly important that they are 
able to identify demands that are either unreasonable or unjustified (Grant, 1999). 

Content and Format 

Large-scale assessments usually are designed to measure students’ mastery of curriculum 
expectations or their performance relative to a norm group. If the test is tied to the curriculum, it 
is especially important that teachers understand the test specifications. Indeed, rumors 
surrounding large-scale assessments often have to do with the item formats and numbers of items 
of each type (Grant, 1999). McGehee and Griffith (2001) observe that teachers will attempt to 
optimize their students’ performance based on what they know about the test. A clear description 
of the proportion of test items of each type should be readily available to counteract unsupported 
speculation. 

Administration 

It may seem obvious, but one of the most important questions about a testing program is, When 
will the test be administered? While principals may provide this information to teachers, posting 
it on the ED web site is helpful, especially when schedules change. 

Another practical issue is how to accommodate students with special needs. EDs have different 
policies about exempting special populations of students or providing modified tests or 
accommodated testing conditions. Teachers need to have access to these policies, especially in 
cases where they are responsible for administering the modified tests or providing the 


accommodations. 


Practice Items and Test-Taking Strategies 

Test preparation often includes, besides teaching the content that is likely to be on the test, 
helping students master test-taking skills and having students answer practice test items or, if 
practice questions are unavailable, questions similar to those that will be used on the test. A 
decade ago, Mehrens and Kaminski (1989), Mehrens (1991), Cohen and Hyman (1991), 
Haladyna, Nolen, and Hass (1991), Popham (1991), and Smith (1991), among others, sought to 
categorize appropriate and inappropriate practices. More recently, Mehrens, Popham, and Ryan 
(1998) proposed guidelines specifically for large-scale performance-based assessments. These 
include the exhortation to “make certain that the student is not surprised, and hence confused, by 
the performance assessments’ format” (p. 20). 

While agreement about appropriate test preparation practices is not unanimous, exposing 
students to the item formats that will be used on the test is widely supported. These may be 
sample items from previous years of the test or practice items that have never been included on a 
test form. Releasing some of the items after a test is administered provides an invaluable 
resource to teachers to help them interpret the results from the previous year and prepare students 
for the upcoming year. Compared to practice items created specifically for use in preparing 
students, released items have greater credibility and may, in fact, be of higher quality since they 
were used on an operational test. 

Consequences 

Will students who do poorly on the test be denied promotion? Will they be required to attend 
summer school? Should they be? Will schools be rewarded if their students perform well and, or 
penalized if they perform poorly? Because different states or provinces have different 
consequences attached to their assessment programs and these consequences may change over 
time, it is critically important that the potential consequences of the test results are clearly 
defined and that teachers are provided with reliable, up-to-date information. 

Using and Explaining Results 

If a parent has a question about test results, he or she will most likely ask the child’s teacher. 
Parents may have some of the same questions about the purpose, content, and format of the test 
that teachers have. Having these questions answered by the teacher may help the parents 
understand their child’s results. In addition, a teacher may be able to explain the limitations of 
the test’s format for students with limited English proficiency or special needs. Finally, teachers 
may need to be prepared to explain more technical issues, such as the meaning of confidence 
intervals on score reports. 

A further concern involves the interpretation of test data relative to other lines of evidence. 
Teachers are aware that test information may be used in connection with other sources of 
evidence (Salmon-Cox, 1981). However, integrating information from large-scale tests with that 
from conventional classroom assessments can be difficult. As a result, teachers are likely to 


benefit from information about appropriate and inappropriate uses of test results. 

Technical Quality 

Information about the validity, reliability, and bias of a test is important to anyone who must 
interpret test scores. That test developers should provide this information is emphasized in the 
Standards for Educational and Psychological Testing (AERA, APA, NCME, 1999). As teachers’ 
training in assessment may not have included technical vocabulary, however, EDs face the added 
challenge of communicating the relevant information without relying on jargon or assuming that 
teachers will have the expertise to judge statistical indices. EDs need to accompany technical 
information with clear conceptual explanations. 

Results from Previous Years 

The availability of assessment results on the Internet has the potential to encourage data-driven 
school improvement initiatives. Recent studies have illustrated how assessment results can be 
used in planning school improvements (Nonce, Perea, & Tracer, 2000; Nicholls & Singer, 2000). 
One article on data-driven school reform has gone so far as to claim that “any district can expect 
gains in student achievement over time when it becomes data-driven” (Nonce et al., 2000, p. 56). 

What Student Testing Information Do EDS Provide? 

In the previous section, we outlined the information EDS should provide. In this section, we ask, 
What information do their web sites provide? Because providing the information is not simply a 
matter of having the information available somewhere on the site, we also examine how easy it is 
to find the information and consider the features of the most and least informative sites. 

Method 

The web sites were surveyed between March and June of 2002 by a team of three reviewers. The 
reviewers were educators with advanced degrees and an interest in assessment. They 
collaborated in developing the protocol for reviewing the web sites, which involved completing a 
chart for each site summarizing its content. Reviewers looked for specific information on each 
web site. Because a teacher visiting a ED web site would likely be looking for information about 
the tests to be taken by a specific grade level, the reviewer was required to select a specific grade 
level. Many states have assessment programs that measure student performance at various grade 
levels and in different subject areas. To simplify the selection, the reviewer was told to check 
first whether a mathematics assessment was administered in Grade 4, 5, or 6. If so, the math test 
for the appropriate grade was the focus of the subsequent search (math was selected because 
most states had math tests in that grade range, and we found that the math tests tended to include 
a wider range of item types than reading or writing tests). If not, another assessment given in 
Grade 4, 5, or 6 was selected. 

While searching the site, the reviewer completed a form that listed information teachers might 
want to know about what to know before the test (purpose, content and format, administration, 
practice items and test-taking strategies) and about what to do when the results were returned 


(consequences, using and explaining results, technical quality, and results from previous years). 
For each piece of information, the reviewer made two judgments: (1) whether the information 
was available and (2) how easy it was to find. In addition, the reviewer also noted the types of 
items used for the assessment (multiple-choice, short answer, extended constructed response, or 
performance tasks) and whether a specific link exists for teachers. Reviewers also took notes 
about exceptional qualities of each site. 

Each reviewer was assigned between 10 and 30 web sites to evaluate. To ensure comparability 
across reviewers, six sites were evaluated by more than one reviewer. For these sites, the 
evaluations were very similar across reviewers; any differences were discussed and differences 
in procedure corrected. Reviewers were told to spend no more than one hour evaluating a given 
site. One hour proved sufficient to thoroughly review most sites; the few sites that could not be 
thoroughly reviewed in an hour would likely present significant usability issues for most site 
visitors. 

The Education Department or Ministry of Education web sites for the Canadian provinces, the 
American states and the District of Columbia were reviewed. (The appendix lists the web sites.) 
Because testing in the state of Iowa was determined at the local level rather than the state level, 
Iowa was not included in the results. The provinces of Manitoba, Newfoundland and Labrador, 
Prince Edward Island, Quebec, and Saskatchewan did not have mandatory large-scale 
assessments in the elementary grades, so were not included. The results, therefore, are based on a 
review of 55 web sites (49 states, 5 provinces, and the District of Columbia). 

Results 

Availability of and ease in finding information. The results of the survey are summarized in 
Table 3 and will be discussed using the categories listed in Tables 1 and 2. Fifty-three of the 55 
sites provided information about the purpose of the test, although 17 of these provided only 
partial information. Of the 53 sites that provided information, the information about purpose was 
easy to find on 28 sites and moderately easy to find on another 18 sites. On the remaining 7 sites, 
it was very difficult to find this information. 

Most sites also provided information about the content of the test (33 available, 14 partially 
available), the types of items (40 available, 5 partially available), when the test would be 
administered (41 available, 2 partially available), and what accommodations could be made for 
students with special needs (35 available, 8 partially available). These pieces of information were 
easy to find on only about half of the sites, however. 

Slightly more than half of the sites provided items from previous years (28 available, 4 partially 
available). Items developed specifically to provide students with practice before taking the test 
were available on 23 sites (20 available, 3 partially available). Fifteen of the sites provided both 
practice items and items from previous years. Only 13 sites provided examples of student work 
(11 available, 2 partially available), but most of these were moderately easy to find (3 easy, 7 
somewhat easy, 3 difficult). Twenty sites (12 available, 9 partially available) provided help for 
teachers wanting to prepare students to use test taking strategies, but only 13 (7 available, 6 
partially available) addressed what types of test preparation are appropriate. 


Most of the sites (44 available, 2 partially available) provided information about how individual 
schools and districts performed in the most recent test administration. In general, this 
information was easy to find (34 easy, 9 somewhat easy, 3 difficult). Almost the same number of 
sites (34 available, 6 partially available) provided some information about performance in 
previous years. However, fewer than a third provided information about the consequences of the 
test results for students (5 available, 13 partially available) and for schools (5 available, 10 
partially available). A similar number provided tips for communicating with parents (5 available, 
11 partially available) and guidelines for appropriate uses of the results (2 available, 8 partially 
available). 

Finally, 21 of the sites (11 available, 10 partially available) provided information about the 
technical quality (e.g., validity and reliability) of the test. 

Site organization. Forty-six of the ED sites had a link labeled “Assessment” or something similar 
on the main page, making it easy for someone looking for information about the testing program 
to begin. These links, however, tended to lead to information that had been organized for 
viewing by a variety of audiences. Only 27 of the sites had links specifically designated for 
teachers. 


Discussion 


Availability of Information 

Many of the web sites examined did not have a link specifically for teachers. When compared 
with sites that did provide teacher-links, the sites without such links generally appeared to be less 
concerned about informing teachers about test-related issues. On some sites, collecting teacher¬ 
relevant test-related information was a Herculean task that involved piecing together widely 
scattered pieces of information. Even some of the sites with teacher-specific links did not 
provide test-related information through those links. 

A few ED sites were designed with a clear consideration of teachers’ needs. The state of Texas 
did a particularly good job of addressing teachers’ concerns. In addition to transmitting important 
information, the Texas site conveyed an impression that teachers’ roles were respected and 
valued. 

Ease in Finding Information 

Grant (1999) found that teachers have limited tolerance for receiving information about testing 
from multiple sources, and ultimately may find it easier to believe what colleagues say than to 
try to evaluate and piece together the various bits of information. Inconsistent messages, even 
from EDS, add to the frustration. 

One of the decisions an ED that uses a web site to provide information to teachers must make is 
how to organize the information. A web site may seem less daunting than the equivalent 
information presented in a large paper document. However, if it is difficult to navigate the site, 


teachers may be discouraged from using it. 

The following properties of the surveyed sites were found by the reviewers to make it easier to 
locate test-related information: 

Information was presented in a logically sequenced manner rather than being scattered. 
Information was presented in a hierarchically-organized structure rather than in lengthy lists. 
Information was in HTML format rather than in PDF documents (PDF documents must be 
downloaded and opened before they can be read; for teachers with slow Internet connections, 
this can make accessing information in PDF documents difficult). 

The test-related link was clearly labeled. 

The site was organized into a few categories. 

The site was structured so that information progressed from general to specific. 

What characteristics should a ED web site have? It should be comprehensive, regularly updated, 
logically laid out and easy to use. It should include a section designed to meet teachers’ needs for 
information. It should include all the information surveyed above. Only a few ED sites have all 
of these characteristics. The following sites, however, were exemplary, having most or all of the 
attributes mentioned above: 

Kansas: Provided a notable amount of information related to special populations. 

Louisiana: Provided data in a spreadsheet format, so that schools can conduct additional analyses 
of test results. 

Texas: Had neither too much nor too little; was very teacher-friendly. 

Hawaii: Described the rationale for the test. 

Colorado: Allowed easy access to the information; the list of assessment-related items was not 
cumbersome. 

Minnesota: Most of the key elements were one or two links away from the main page; very well 
organized in an easy-to-use hierarchical manner. 

Florida: Included a section entitled What Every Teacher Should Know; offered lots of 
information organized with teachers in mind. 

Delaware and Maine: Included very useful graphics organizing the links to test-related 
information. These graphical tools give the user an overview of the site and an organizer for 
searching for information. 

Limitations 

One of the advantages of web sites is that they are dynamic. That is, they can be frequently 
updated and redesigned and the new versions will be immediately available to users. This study 
is, necessarily, therefore, a snapshot of ED web sites in the spring of 2002. Some of the sites may 
already have improved since we reviewed them. 

In addition, most of the web sites were reviewed only once. Because the sites evolved, repeat 
reviews had to be performed over a short period of time to be useful for checking interrater 
consistency. Therefore, although the three reviewers collaborated in developing the questions 
that were answered in surveying the web sites and discussed decisions made in answering them, 


interrater reliability was not formally analyzed. Given that this study describes the range of 
practice, rather than reporting an evaluation of each jurisdiction’s site, this should not limit the 
usefulness of the findings. 

Finally, making information available and easy to find is only helpful to teachers if the 
information is of good quality. Verifying the accuracy and completeness of the information 
provided on ED web sites would require checking against other sources of information, which is 
beyond the scope of this study. A study of fewer sites, comparing the information provided with 
other sources of information would be a useful follow-up. 

Challenges for the Future 

The Internet provides opportunities for going beyond simply providing information. For 
example, a site could provide a venue for teachers to give feedback about the tests. Such a site 
would permit teachers to become involved with and inform the assessment process. EDS have 
the opportunity to create very useful state-of-the-art web-based tools for teachers that are a 
source of both information and empowerment. 

In addition to providing opportunities for feedback, ED sites could provide teachers with 
alternative views of the testing process. The benefit of including such information would be to 
allow substantive issues to be discussed. The immediate need, of course, is to provide teachers 
information about testing programs; once that is accomplished, however, we hope ED sites will 
consider actively engaging teachers in conversations about testing. 

Discussion 

This study identified eight categories of information teachers need to know about large-scale 
testing programs and examined Education Department and Ministry of Education web sites to 
see whether such information was available and easy to access. While many sites presented 
extensive information about the testing programs, only a few tailored the information to the 
needs of teachers. Our hope is that these results will encourage more effective use of web sites as 
communication channels between Education Departments and teachers. 
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NOTES 

1. Values in the table are numbers of web sites. The question “Is it easy to find?” is not 
applicable to web sites for which the answer to “Is it available?” is “no.” 


T able 1 


Supportfor Categories of Information About Student Testing Programs 




Table 2 


Questions ED Web Sites Should Answer for Teachers 


Category 

Question 

Purpose 

Why is the test being administered? 

Content & Format 

What knowledge and skills will the test measure? 


What types of items will the test include? 

Administration 

When will the test be administered? 


Will accommodations be available for students with special needs? 

Practice Items 
& T e st-T aking Strate gie s 

Where can I find sample items (from previous years)? 


Where can I find practice items (developed specifically to help students practice 
for the test)? 


Where can I find examples of student work from previous years? 


Where can I find example scoring guides? 


What test-taking strategies might help my students when they take the test? 


What test preparation activities are appropriate? 

Consequences 

What consequences will the test results have for my students? 


What consequences will the test results have for schools? 




















Table 3 


Availability of and Base in Finding Information on ED Web Sites 


Question 

Is It Available? 


Is It Easy to Find? 1 


Yes 

Partially 

No 

Yes 

Somewhat 

No 

Purpose 






Why is the test being administered? 

36 

17 

2 

28 

18 

7 

Content & Format 








What knowledge and skills will the test measure? 

33 

14 

8 


16 

22 

9 

What types of items will the test include? 

40 

5 

10 


20 

20 

5 

Administration 








When will the test be administered? 

41 

2 

12 


27 

12 

4 

Will accommodations be available for students with 
special needs? 

35 

8 

12 


23 

16 

4 

Practice Items & Test-Taking Strategies 








Where can I find sample items (fromprevious 
years)? 

28 

4 

23 


19 

9 

4 

Where can I find practice items (developed 
specifically to help students practice for the test)? 

20 

3 

32 


11 

9 

3 

Where can I find examples of student work from 
previous years? 

11 

2 

42 


3 

7 

3 

Where can I find example scoring guides? 

20 

11 

24 


12 

11 

8 

What test-taking strategies might help my students 
when they take the test? 

12 

9 

34 


7 

10 

4 














































APPENDIX 


Education Department/Ministry of Education Web Sites Reviewed 


URL 

Grade 

Subject 

Link for 
Teachers 

Link for 
Program 

Province 






Alberta 

http://www.learning.gov.ab.ca 

6 

Math 

X 

X 

British Columbia 

http://www.gov.bc.ca/bced 

4 

Math 

X 

X 

Manitoba 1 

http://www.edu.gov.mb.ca 





New Brunswick 

http://www.gnb.ca/0000/index-e.asp 

5 

Math 

X 

X 

Newfoundland and 
Labrador 1 

http://www.gov.nf.ca/edu 





Nova Scotia 

http://www.ednet.ns.ca 

5 

Math 


X 

Ontario 

http://www.edu.gov.on.ca 

6 

Math 


X 

Prince Edward Island 1 

http://www.edu.pe.ca 





Quebec 1 

http://www.meq.gouv.qc.ca 





Saskatchewan 1 

State 

http://www.sasked.gov.sk.ca 





Alabama 

http://www.alsde.edu 

5 

Writing 



Alaska 

http://www.educ.state.ak.us 

6 

Math 

X 

X 

Arizona 

http://www.ade.state.az.us 

5 

Math 

X 

X 

Arkansas 

http://arkedu.state.ar.us 

4 

Math 


X 

California 

http://www.cde.ca.gov 

5 

Math 


X 

Colorado 

http://www.cde.state.co.us 

5 

Math 

X 

X 
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